1. Field of the Invention
This invention relates to digital photogrammetric processes and more specifically to a method of photogrammetric image mensuration in which grid marks can be provided, yet can be made invisible to an image of the object to be mensurated and analyzed.
2. Background of the Invention
The photographic art for aerial surveillance, geological and archaeological study, mechanical, industrial, and architectural design and analysis, and other uses has become very well-developed over the past several decades so that sharp, clear photographic images of the earth's surface and of objects on the earth's surface are obtainable from aerial photography, satellite photography, and the like. In fact, there already are in existence virtually countless aerial photographs in files of national, state, and local government agencies, corporations, and individuals for purposes ranging widely from such things as military reconnaissance, surveillance and measurement of agricultural land and crop conditions, monitoring municipal development and growth patterns, map making, geologic assaying, land management, and the like. Additional photographing and rephotographing for subsequent comparison with previous conditions are being done on an increasing basis.
For many purposes, however, analysis of such photographic images cannot be done by visual observation with sufficient accuracy or efficiency. For example, in spite of having exceptionally clear aerial photographic images available, it may be quite impossible, even with accurate graphic instruments and a magnifying glass, to measure the wing-span of an airplane parked on an airport apron, the square feet of pavement on all of the streets in a city, or the areas of potholes in a wetlands inventory of a prairie.
Therefore, to improve their accuracy and efficiency, persons skilled in the art of photogrammetry have found that computers can be a very useful tool for enhancing the photographic images or parts of the images and to augment the analysis. To do so, the photographic image is converted into a digital format that can be stored, processed, and displayed on a computer controlled graphic display output, such as a cathode ray tube (CRT), hard copy printer, plotter, or the like.
A common method of converting a hard copy image to a digital array is to use a point sensor, such as a charge-coupled device (CCD), charge injection device (CID), or photodiode to scan the surface of the hard copy and measure the light either transmitted through, or reflected from, various points on the hard copy. The hard copy in this kind of process is usually mounted on a rotating drum or on a flat table that is movable in orthogonal X-Y directions. A large pixel array, such as a 20,000 by 20,000 pixel area, can be acquired, which may, for example, be the pixel array needed to represent the information on a 9".times.9" (23 cm.times.23 cm) film image, assuming individual pixels of about 12.5 um diameter.
Some systems use a linear detector or sensor array, instead of a single point sensor for the digital data acquisition. In such linear systems a large number (e.g., 1750) of individual light sensitive elements are grouped together in a linear row, and this linear array or row of sensors is used to sweep scan a path over the surface of the hard copy.
Precise mechanical motion control is required for both the individual scan lines of a single point sensor and the groups of scan lines or sweep path of linear arrayed sensors in order to obtain a meaningful and useable pixel array of the photogrammetric image. Such mechanical accuracy, while necessary for accurate pixel designation and resolution, cannot be obtained economically in the degree that would be required for resolution commensurate to pixel sizes of less than, for example, 50 microns. Also, typical operations problems with such systems usually result from inability to achieve and maintain the mechanical accuracy needed over long periods of time. Consequently, the large data arrays required and the high cost to obtain the necessary mechanical accuracy have kept the use of digital image processing of photographic images in laboratories only and away from general commercial application and use.
In recent years, several manufacturers have made available semiconductor chips on which a plurality of CCD's or CID's are arranged in a two-dimensional, rectangular array and mounted in a solid state camera, such as a "TM-540", manufactured by Pulnix, of Sunnyvale, Calif. These solid state cameras with rectangular sensor arrays can detect and measure light from a fixed frame or rectangular portion of the image that a person desires to digitize for computer use. When such cameras are used in conjunction with an analog to digital converter (sometimes called a "frame grabber" device), the signal point or linear array scanning is no longer required to acquire a pixel array of digital values for a photogrammetric computer image of a hard copy photograph, transparency, drawing, or the like. The physical spacings and sizes of the pixels are fixed by the geometric CCD or CID array and by the magnification of the hard copy image to the CCD or CID array.
These "frame grabbing" solid state cameras typically have rectangular arrays, such as, for example, about 510.times.492 CCD's or CID's. When properly focused on an image, each CCD or CID in the array detects light intensity from an individual spot or pixel area on the film image. Thus, a solid state camera that has an array of 510.times.492 CCD's on a rectangular chip will convert the portion of a film image within a focused frame to a square pixel array of 510.times.492, i.e., about 250,920 light intensity measurements or signals. Such an array of intensity measurements can, of course, be recorded and displayed by a computer on a CRT in the same pixel array to provide a computer image reproduction of the portion of the film image within the focused or "grabbed frame". There has been a recent announcement by at least one manufacturer that a solid state CCD camera with a 1,000 .times.1,000 pixel array will soon be available, which will provide larger "grabbed" frames, more accuracy, or a combination of both.
While the "frame grabbing" solid state cameras with rectangular CCD or CID arrays eliminate scanning, as described above, they are applicable only where a limited size pixel array is needed. For example, such a "frame grabber" may be useful in focusing onto, and acquiring a digital image of, a particular small object, such as an airplane, that can be seen in an aerial photograph of a ten square kilometer area. However, they have not been useful before this invention for "grabbing" and digitizing larger film image areas. In order to "grab" and digitize a larger film image area, the solid state camera had to be focused over a larger film area, thus sacrificing detail accuracy, since each pixel size within the array also is focused over a larger area.
There are at least two products now available that can create a large pixel array by combining a "frame grabbing" two-dimensional image array with a scanning motion. In such systems, individual frames or sub-areas of larger macroareas of film or paper photographs can be "grabbed" or digitized and stored. Then, adjacent frames can be "grabbed" and positioned correctly in the computer memory by either (1) moving the "frame grabbing" solid state camera very precisely to a predefined adjacent position mechanically and then "grabbing" the pixel array for that adjacent position, or (2) by moving the camera less precisely to "grab" the image at the adjacent location and 5 relying on a precisely located grid mark or pattern of grid marks to geometrically relate one "grabbed" sub-area to the next "grabbed" sub-area. The "Autoset-1" manufactured by Geodetic Services Incorporated, of Melbourne, Fla. is an example of the former of these techniques, and the "Rolleimetric RS", manufactured by Rollei Fototechnic GmbH, of Braunschweig, West Germany, is an example of the latter technique.
In general, reasonably priced opto-mechanical scanners have not been able to achieve the accuracy considered to be necessary for many of the newly-evolving applications. Scanners that could achieve high geometric resolution are slow and often force a user to resort to an off-line scanning process separate from the process of actually using and analyzing the data.
Frame grabbing solid state camera systems, as described above, provide a higher degree of accuracy within a small frame pixel array sub-area. However, combining frame grabbing with scanning to get digital data over a larger macro-area again usually sacrifices accuracy for economy or economy for accuracy due to the need for highly accurate mechanical position control. The Rollei system mentioned above, and further described in the West German patent no. DE 3428325, is considered to be a significant advancement in this regard by teaching the use of reseau grids in combination with a "frame grabbing" solid state camera, but it still requires manual identification of reseau grids or crosses. Also, the reseau crosses or grids are visible in the image and obliterate some of the contents of the photographic image where the grid marks are located. Also, the process of using a reseau in that manner is slow.
SUMMARY OF THE INVENTION
Accordingly, it is an object of the present invention to provide a fast, accurate, yet relatively inexpensive image digitizing and mensuration system for analyzing hard copy photographic, transparency, paper drawing, radar, and other images.
A more specific object of the present invention is to provide an improved mensuration frame grabbing system for digitizing and analyzing hard copy photographic, transparency, paper drawing, radar, and other images.
A still more specific object of the present invention is to provide an improved reseau grid for a mensuration frame grabbing system in which the reseau grid does not obscure or cover any part of the image and does not become a part of the image.
Another specific object of the present invention is to provide a frame grabbing digitizing mensuration system that uses a reseau grid location reference system in which individual reseau detection and location is automatic.
Still another specific object of this invention is to provide an image digitizing system in which one or more specific sub-areas of a large macro-area image can be converted to digital format without having to convert the entire macro-area image to digital format if not desired, thus avoiding the use of unnecessary computer storage and off-line creation of a large pixel array and allowing mass storage of currently uninteresting image to be kept on film only, yet which also has the capability of digitizing an entire large format macro-area image, if desired, in an efficient, accurate manner.
A further specific object of the present invention is to provide a system that can quickly and accurately digitize a select feature shown in stereo photographs, correlate the digital images, and display them in a stereo image, such as a three-dimensional display or other stereo overlapping images, on a CRT, graphic display device, or the like.
A still further object of the present invention is to provide a relatively inexpensive, compact apparatus for digitizing and analyzing hard copy images in which all parts of a large hard copy image are kept visible and stationary at all times.
Additional objects, advantages, and novel features of this invention are set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following specification or may be learned by the practice of the invention. The objects and advantages of the invention may be realized and attained by means of the instrumentalities and in combinations particularly pointed out in the appended claims.
To achieve the foregoing and other objects and in accordance with the purposes of the present invention, as embodied and broadly described herein, the method and apparatus of this invention includes the steps of positioning below a solid state camera a reseau with grid marks thereon adjacent an object with an image and activating one of several grid mark visibility means. Digitizing the grid marks and storing the digitized data including spatial location in a computer memory. Removing the grid marks from the capability of the solid state camera to view them and digitizing the image on the object and storing the digitized data of the object in a computer memory. Correlating the digitized data of the grid marks with that of the image of the object for the purpose of computer controlled visual display and/or mensuration of the image by the computer, with respect to a scale calibrated by the computer from the digitized reseau grid marks.
There are several methods and associated apparatus for making the grid marks visible at one time and then invisible at another according to this invention. One such method is to fit the solid state camera with a gain control and to make the reseau grid mark visible by illumination with high intensity light, such as Xenon light. The intensity of Xenon light makes it possible to have the grid marks and object illuminated simulataneously and still allow the camera to distinguish and digitize the grid marks. The invisibility of the marks occurs when the Xenon light is removed. Another method is to have the reseau grid formed from a light polarizing material and making the reseau grid marks visible by passing light polarized perpendicular to the polarizing material. The grid marks become invisible when unpolarized light or light polarized parallel to the polarizing material is allowed to pass through the reseau. A third method is having the reseau grid formed on an electrochromic active film on the reseau plate, whereby the reseau grid marks are made visible by placing an electric potential across the reseau. Conversely, the marks are made invisible by removing the electric potential from across the reseau. A fourth method includes filtering the solid state camera with a filter that includes a clear, colored, or infrared filter and making the reseau grid a specific color or infrared by illumination with light of that wavelength or deposition of a material that either reflects light of that wavelength or allows light of that wavelength to pass through. The reseau grid marks are made visible by illumination of the reseau, and placing a filter not corresponding to the wavelength transmitted by the grid marks before the camera. Conversely, the grid marks are made invisible by placing the colored or infrared filter before the camera, corresponding to the wavelength transmitted by the grid marks. A fifth method includes separating the reseau grid from the object with sufficient distance that the solid state camera's focal length and depth of field allows the camera to focus on only the reseau grid marks or the object but not both. The grid marks can be made visible by adjustment of the camera's focus such that the grid marks are in focus and the image in the object is essentially washed out. Conversely, the grid marks can be made invisible by adjusting the focus such that the grid marks are washed out and the image on the object is in focus.